Learning and Visual Hallucination

It is generally recognized that learning is a useful tool for solving vision problems. However, at present time, neither learning nor vision is a well understood problem. It is even less clear how one might be able to put the two together. In this paper I will attempt to study one small aspect of this problem of using learning in vision. In particular I will study how learning might be used for visual hallucination. By visual hallucination, we mean the ability to take a visual input and reconstruct from the memory a similar image(in the same or diierent representation). The hope is that the hallucinated images are easier to reason about, and we can operate in this hallucinated world rather that the real world. In some sense this is the basic idea behind the model based object recognition. To make the problem more concrete, I will look at how such visual hallucination can be used in the problem of locating cars in the traac scene. Figure 1 shows one typical traac scene I am working on. As we can see from gure 1, one big headache is the shadows casted by the cars. DM94] has reported one approach for locating cars using motion information. However, we can not locate the car using the motion information alone for the images such as gure 1, since the shadows move with the cars. In this situation, some kind of car recognition algorithm is needed for distinguishing car from shadow. I have developed one pure vision technique for solving this problem, as reported in my CS280 class project. In this learning project, I will study how one particular learning technique, manifold learning, can also be used for car recognition. The basic idea is as follows. We rst present the learning agent a large number(500) of images of cars without the shadows. The learning agent then constructs an internal representation of the car it has seen so far. In the testing phase, we present the image of a car along with the shadow to the agent. The agent then reconstructs a hallucinated image from its internal representation that is the same as the input image except no shadow is present. We can then nd the bounding quad. on the hallucinated image, and map it back onto the input image. It might seem to be a lot of wishful thinking. But as I will show in …